Motor vehicle warning and control system and method

ABSTRACT

A system and method assists the driver of a motor vehicle in preventing accidents or minimizing the effects of same. In one form, a television camera or other ranging device is mounted on a vehicle and scans the roadway ahead of the vehicle as the vehicle travels. Continuously generated video picture signals output by the camera are electronically processed and analyzed by a fuzzy-logic-based image analyzing computer mounted in the controlled vehicle, which generates control signals and applies them to control the operation of the accelerator, brake, and steering system of the vehicle in a coordinated way to attempt to avoid or lessen the effects of a collision. In a particular form, the decision computer may select the evasive action taken from a number of choices, depending on whether and where the detection device senses other vehicles and obstacles. Warning signals may also be generated.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 08/671,853,filed Jun. 28, 1996, now U.S. Pat. No. 6,553,130 which is a continuationof application Ser. No. 08/105,304, filed Aug. 11, 1993, now abandoned.

FIELD OF THE INVENTION

This invention relates to a system and method for operating a motorvehicle, such as an automobile, truck, aircraft or other vehicle,wherein a computer or computerized system is employed to assist and/orsupplement the driver in the movement of the vehicle along a path oftravel, such as a street or roadway and may be used to avoid obstaclesand accidents.

BACKGROUND OF THE INVENTION

A major cause of human suffering is automobile accidents. Approximately49,000 people die in traffic accidents each year in the United States,and another three million are injured. The costs of death and injuryaccidents are staggering. According to the United States NationalHighway Traffic Safety Administration, crash damage and medical billstotal $137 billion a year.

Automobile designers offer many safety features, including passengerrestraints, improved braking systems, and body designs, intended tobetter protect automobile crash victims. But very little has been donein the area of automatic vehicle control systems based on modernelectronics, computer systems, and advanced real-time software. This istrue despite rapidly increasing capabilities in these technologies andpervasive application in many other areas including, for example thebusiness, entertainment, and medical fields. Vehicle guidance andcontrol technology has, of course, been applied with great success inmilitary defense systems, avionics systems and space explorationsystems. But, this technology is costly and has not been commercialized.

The opportunity exists today to develop cost effective, commercialautomated vehicle control systems. New advances in low-cost hardware andsoftware technology make implementation feasible. High-speed, parallelcomputer architectures, specialized image-processing equipment, andadvanced special computers such as math co-processors are available.Advanced expert system implementations based on concepts such as fuzzylogic and neural networks, and new, improved scanning systems forsensing environments around moving vehicles make it very timely, indeed,to pursue new approaches.

Work on these problems has begun. Intelligent vehicle/highway systemsare being investigated with traffic control systems intended to minimizecongestion. Vehicle location systems such as GPS (Global PositioningSystem) and route guidance systems are also being pursued. Certainsystems for automated vehicle control have been proposed, includingsystems that scan the roadway directly ahead of a vehicle usingradar/lidar or television and attempt to warn a driver of impendingdanger. Fuzzy logic expert systems for controlling vehicle speed(braking and throttle) based on scanning the roadway ahead of a vehiclehave been described. Road tracking with electronic vehicle guidance isbeing pursued. Fuzzy logic has been applied to braking systems in subwayand train systems.

While these developments are important, they fail to protect vehiclesfrom many types of collisions or minimize the damage therefrom. Moreparticularly, such systems fail to exercise simultaneous, coordinatedcontrol over vehicle steering and speed, fail to take full advantage ofidentification of different obstacle or hazard types using standardstored models of production vehicles and other commonly encounteredroadway objects, fail to deal effectively with objects and hazardslocated simultaneously on different sides of the vehicle, and fail tocapitalize fully on modern expert system decision and controltechnology, such as represented by fuzzy logic and neural networkmethods, to deal with more complex hazardous situations.

SUMMARY OF THE INVENTION

In a preferred form of the invention, a video scanning system, such as atelevision camera and/or one or more laser scanners mounted on thevehicle scan the road in front of the vehicle and generate imageinformation which is computer analyzed per se or in combination with arange sensing system to warn the driver of hazardous conditions duringdriving by operating a display, such as a heads-up display, and/or asynthetic speech generating means which generates sounds or words ofspeech to verbally indicate such road conditions ahead of the vehicle.

The preferred form of the invention provides audible and/or visualdisplay means to cooperate in indicating to the driver of a motorvehicle both normal and hazardous road conditions ahead as well asdriving variables such as distances to stationary objects, and othervehicles; the identification, direction of travel and speed of suchother vehicles and the identification of and distances to stationary orslowly moving objects such as barriers, center islands, pedestrians,parked cars poles, sharp turns in the road and other conditions. Inaddition, the image analyzing computer of the vehicle may be operated toscan and decode coded and/or character containing signs or signalsgenerated by indicia or code generating other devices within or at theside of the road and indicating select road and driving conditionsahead.

The computer is operable to analyze video and/or other forms of imageinformation generated as the vehicle travels to identify obstacles aheadof the vehicle and, in certain instances, quantify the distance betweenthe vehicle containing same on the basis of the size of the identifiedvehicle or object and/or by processing received pulse-echo signals.Using such identifying information and comparing it with information onthe shapes and sizes of various objects such as rear and front profilesof all production vehicles and the like and their relative sizes orselect dimensions thereof, indications of distances to such objects maybe computed and indicated as further codes.

When the closing distance becomes hazardous, select vehicle subsystemsmay be automatically controlled by the computer as it continues toanalyze image signals generated by the television camera. A firstsubsystem generates a first select code or codes which controls anelectronic display, such as a heads-up display to cause it to display awarning indication, such as one or more flashing red light portions ofthe display or other lighted effect. For example, the display mayproject on the windshield or dashboard such information as images of thecontrolled vehicle and other vehicles in and adjacent its path of traveland relative distances thereto as well as groups of characters definingsame, colored and flashing warning lights and the like for pre-warningand warning purposes. A second subsystem generates a code or series ofcodes which control a sound generating means which generates a selectsound such as a horn, buzzing sound and/or select synthetic speechwarning of the hazardous condition detected and, in certain instances,generating sounds of select words of speech which may warn of sameand/or suggest corrective action(s) by the vehicle operator or driver toavoid an accident.

A third subsystem comes on-line and generates one or more codes whichare applied to at least partly effect a corrective action such as bypulsing one or more motors or solenoids to apply the brakes of thevehicle to cause it to slow down. If necessary to avoid or lessen theeffects of an accident, the third subsystem stops the forward travel ofthe vehicle in a controlled manner depending on the relative speeds ofthe two vehicles, and/or the controlled vehicle and a stationery objector structure and the distance therebetween.

A fourth subsystem, which maybe part of or separate from the thirdsubsystem may generate one or more codes which are applied to eithereffect partial and/or complete control of the steering mechanism for thevehicle to avoid an obstacle and/or lessen the effect of an accident.Either or both the third or fourth subsystem may also be operable tocontrol one or more safety devices by controlling motors, solenoids orvalves, to operate a restraining device or devices for the driver andpassenger(s) of the vehicle, such as a safety belt tightening means, anair bag inflation means or other device designed to protect human beingsin the vehicle.

The second, and/or third and fourth subsystems may also be operable toeffect or control the operations of additional warning means such as thehorn, headlights and/or other warning lights on the vehicle or otherwarning means which operates to alert, flag or warn the driver of theapproaching or approached vehicle or a pedestrian of the approachinghazardous condition. One or more of these subsystems may also beoperable to generate and transmit one or more codes to be received andused by the approaching or approached vehicle or a roadside device toeffect additional on-line warning(s) of the hazardous condition, and/ormay be recorded on a disc or RAM (random access memory) for futureanalysis, if necessary.

In a modified form of the invention, the vehicle warning system may alsoinclude a short wave receiving means to receive code signals from othervehicles and/or short wave transmitters at the side of or within theroad for controlling the visual, audio and/or brake and steering meansof the vehicle to avoid or lessen the effects of an accident and/or tomaintain the vehicle in-lane and in proper operating condition as ittravels.

The systems and methods of this invention preferably employ computerizedimage analyzing techniques of the types disclosed and defined in suchpatents of mine as U.S. Pat. Nos. 4,969,038 and 4,979,029 and referencescited in the file wrappers thereof as well as other more recent patentsand include the use of known artificial intelligence, neural networkingand fuzzy logic computing electronic circuits.

While the invention is described herein principally in connection withan automobile on a roadway, it may be used in connection withcontrolling any powered vehicle, including a motor vehicle, a boat, atrain, or an aircraft.

Accordingly it is a primary object of this invention to provide a newand improved system and method for controlling the operation of apowered vehicle.

Another object is provide a system and method for assisting the driverof a powered vehicle in controlling its operation to avoid an accidentor hazardous driving condition.

Another object is to provide a system and method employing computerizedimage analysis to control or assist the driver of a motor vehicle incontrolling its operation to avoid hazardous conditions such ascollisions with other vehicles, stationery objects or pedestrians.

Another object is to provide a computerized system and method forcontrolling the speed of travel of a motor vehicle to lessen the chancesof an accident while being driven by a person.

Another object is to provide a system and method employing a televisionscanning camera mounted on a vehicle for scanning the field ahead, suchas the image of the road ahead of the vehicle and a computer foranalyzing the image signals generated wherein automatic imageintensifying, or infra-red scanning and detection means is utilized topermit scanning operations to be effected during driving at night and inlow light, snowing or fog conditions.

Another object is to provide a system and method employing a televisioncamera or other video scanning means mounted on a moving motor vehiclefor scanning, detecting and identifying obstacles such as other vehiclesahead of such moving vehicle wherein the video image signals areanalyzed to determine distances to such objects.

Another object is to provide a computer controlled safety system for amotor vehicle which employs a television camera and an auxiliaryscanning means to both identify obstacles in the path of the vehicle anddetermine distance therefrom on a real time and continuous basis for usein warning the operator of same and/or in controlling the operation ofthe vehicle to avoid a collision.

BRIEF DESCRIPTION OF DRAWINGS

The various hardware and software elements used to carry out theinvention described herein are illustrated in the form of blockdiagrams, flow charts, and depictions of neural network and fuzzy logicalgorithms and structures. The preferred embodiment is illustrated inthe following figures:

FIG. 1 is a block diagram of the overall Motor Vehicle Warning andControl System illustrating system sensors, computers, displays,input/output devices and other key elements.

FIG. 2 is a block diagram of an image analysis computer 19 of the typethat can be used in the Vehicle Hazard Avoidance System herein of FIG.1.

FIG. 3 illustrates a neural network of the type useful in the imageanalysis computer of FIG. 4.

FIG. 4 illustrates the structure of a Processing Element (PE) in theneural network of FIG. 3.

FIG. 5 is an alternate embodiment of a neural network image processoruseful in the system of FIG. 1.

FIG. 6 is a flow diagram illustrating the overall operation of the MotorVehicle Warning and Control System of FIG. 1.

FIG. 7 illustrates typical input signal membership functions for fuzzylogic algorithms useful in the Motor Vehicle Warning and Control Systemof FIG. 1.

FIG. 8 illustrates typical output signal membership functions for fuzzylogic algorithms useful in the Motor Vehicle Warning and Control Systemof FIG. 1.

FIG. 9 illustrates typical Fuzzy Associative Memory (FAM) maps for thefuzzy logic algorithms useful in the Motor Vehicle Warning and ControlSystem of FIG. 1.

FIG. 10 is a Hazard/Object state vector useful in implementing the FuzzyLogic Vehicle Warning and Control System.

FIG. 11 is a Hazard Collision Control vector useful in implementing theFuzzy Logic Vehicle Warning and Control System.

FIG. 12 is a table of Hazard/Object state vectors indicating possiblecombinations of hazards and objects useful in the Fuzzy AssociativeMemory access system used herein.

FIG. 13 is a more detailed logic flow diagram for the analysis ofdetection signals prior to accessing fuzzy logic control structures inthe Motor Vehicle Warning and Control System.

FIG. 14 is a more detailed logic flow diagram for the Fuzzy AssociativeMemory (FAM) selection processing.

FIG. 15 is an example system flow illustrating the operation of theMotor Vehicle Warning and Control System.

DETAILED DESCRIPTION

In FIG. 1 is shown a computerized control system 10 for controlling theoperation of a motor vehicle to prevent or lessen the effects ofaccidents such as collisions with stationery and/or moving objects suchas other vehicles. The system 10 employs a control computer ormicroprocessor 11 mounted on the vehicle and operable to receive andgate digital signals, such as codes and control signals from varioussensors, to one or more specialized computers and from such computers toa number of servos such as electric motors and lineal actuators orsolenoids, switches and the like, speakers and display drivers toperform either or both the functions of audibly and/or visuallyinforming or warning the driver of the vehicle of a hazardous roadcondition ahead and/or to effect controlled braking and steering actionsof the vehicle.

A RAM 12 and ROM 13 are connected to processor 11 to effect andfacilitate its operation. A television camera(s) 16 having a wide anglelens 16L is mounted at the front of the vehicle such as the front end ofthe roof, bumper or end of the hood to scan the road ahead of thevehicle at an angle encompassing the sides of the road and intersectingroads. The analog signal output of camera 16 is digitized in an A/Dconvertor 18 and passed directly to or through a video preprocessor 51to microprocessor 11, to an image field analyzing computer 19 which isprovided, implemented and programmed using neural networks andartificial intelligence as well as fuzzy logic algorithms to (a)identify objects on the road ahead such as other vehicles, pedestrians,barriers and dividers, turns in the road, signs and symbols, etc., andgenerate identification codes, and (b) detect distances from suchobjects by their size (and shape) and provide codes indicating same foruse by a decision computer, 23, which generates coded control signalswhich are applied through the computer 11 or are directly passed tovarious warning and vehicle operating devices such as a braking computeror drive, 35, which operates a brake servo 33, a steering computer ordrive(s) 39 and 40 which operate steering servos 36; a synthetic speechsignal generator 27 which sends trains of indicating and warning digitalspeech signals to a digital-analog converter 29 connected to a speaker30; a display driver 31 which drives a (heads-up or dashboard) display32; a headlight controller 41 for flashing the headlights, a warninglight control 42 for flashing external and/or internal warning lights; ahorn control 43, etc.

A digital speedometer 44 and accelerometer(s) 45 provide informationsignals for use by the decision computer, 23, in issuing its commands.Accelerometer(s) 45 are connected to control computer microprocessor 11through analog-to-digital converter 46. The accelerometer(s) 45 may passdata continuously to control computer microprocessor 11, or,alternatively, respond to query signals from said control computer 11.An auxiliary range detection means comprises a range computer 21 whichaccepts digital code signals from a radar or lidar computer 14 whichinterprets radar and/or laser range signals from respective reflectedradiation receiving means on the vehicle. In a modified form, videoscanning and radar or lidar scanning may be jointly employed to identifyand indicate distances between the controlled vehicle and objects aheadof, to the side(s) of, and to the rear of the controlled vehicle.

The image analyzing computer 19 with associated memory 20 may beimplemented in several different ways. Of particular concern is therequirement for high speed image processing with the capability todetect various hazards in dynamic image fields with changing scenes,moving objects and multiple objects, more than one of which maybe apotential hazard. Requirements for wide angle vision and the ability toanalyze both right and left side image fields also exist. The imagingsystem not only detects hazards, but also estimates distance based onimage data for input to the range computer 21 implemented with theassociated memory unit 22.

High speed image processing can be implemented employing known specialpurpose computer architectures including various parallel systemstructures and systems based on neural networks. FIG. 2 shows a highspeed parallel processor system embodiment with dedicated imageprocessing hardware. The system of FIG. 2 has a dedicated image data bus50 for high speed image data transfer. The video camera 16 transfersfull-frame video picture signal/data to the image bus 50 viaanalog/digital converter 18 and video preprocessor 51. The video camera16 is preferably a CCD array camera generating successive picture frameswith individual pixels being digitized for processing by the videopreprocessor 51. The video camera 16 may also be implemented with othertechnologies including known image intensifying electron gun andinfra-red imaging methods. Multiple cameras may be used for front, sideand rear viewing and for stereo imaging capabilities suitable forgeneration of three-dimensional image information including capabilitiesfor depth perception and placing multiple objects in three dimensionalimage fields to further improve hazard detection capabilities.

As shown in FIG. 2, the video preprocessor 51 performs necessary videoimage frame management and data manipulation in preparation for imageanalysis. The preprocessor 51 may also be used in some embodiments fordigital prefiltering and image enhancement. Actual image data can bedisplayed in real time using video display 55 via analog-to-digitalconverter 54. The image display may include highlighting of hazards,special warning images such as flashing lights, alpha-numeric messages,distance values, speed indicators and other hazard and safety relatedmessages. Simulated displays of symbols representing the hazard objectsas well as actual video displays may also be used to enhance driverrecognition of dangerous situations.

The image analysis computer 19 operates under the control of controlprocessor 56 with random-access-memory (RAM) 57 and program andreference data stored in read-only memory (ROM) 58. The controlprocessor 56 communicates with the motor vehicle warning and controlsystem micro-processor controller 11 through the Bus Interface Unit 59.Results of the image analysis are passed in real-time to microprocessorcontroller 11 for integration with other sensory, computing, warning andcontrol signals as depicted in FIG. 1.

The image analysis computer 19 of FIG. 2 uses high speed dedicatedco-processor 53 for actual image analysis under control of the controlprocessor 56. Typical operations performed using co-processors 53include multidimensional filtering for operations such as featureextraction and motion detection. The co-processors 53 are used formultidimensional discrete transforms and other digital filteringoperations used in image analysis. Multiple image memories 52 withparallel access to successive image data frames via image bus 50 permitconcurrent processing with high speed data access by respectiveco-processing elements 53. The co-processor elements 53 may be highspeed programmable processors or special purpose hardware processorsspecifically constructed for image analysis operations. SIMD (singleinstruction, multiple data) architectures provide high speed operationwith multiple identical processing elements under control of a controlunit that broadcasts instructions to all processing elements. The sameinstruction is executed simultaneously on different data elements makingthis approach particularly well suited for matrix and vector operationscommonly employed in image analysis operations. Parallel operations ofthis type are particularly important with high pixel counts. A 1000×1000pixel image has one million data points. Tightly coupled MultipleInstruction, Multiple Data (MIMD) architectures also are used in imageprocessing applications. MIMD systems execute independent but relatedprograms concurrently on multiple processing elements. Various arrayprocessor and massively parallel architectures known to those skilled inthe art may also be used for real-time image analysis.

The calculation of the distance of certain recognizable objects from thevehicle is facilitated by having standard images stored in memory andrecalling and comparing such image data with image data representing theobject detected by the vehicle scanning mechanisms. For example,virtually all automobiles, trucks, and other standard vehicles haveknown widths. It follows that the distance to a second powered vehiclesuch as an automobile or truck can be determined by calculating itswidth in the scanned image. If a CCD camera is used, for example, thewidth can ascertained in pixels in the image field. The distance to thevehicle can then be easily calculated using a simple relationshipwherein the distance will be directly proportional to the object imagewidth in pixels. The relative velocities and accelerations can also beeasily calculated from respective first and second derivatives of theimage width with respect to time. These image measurements andcalculations can be used in addition to radar/lidar signal measurementsor they may be used alone depending on system requirements.

In another embodiment, the image analyzing computer 19 is implemented asa neural computing network with networked processing elements performingsuccessive computations on input image structure as shown in FIG. 3where signal inputs 61 are connected to multiple processing elements 63,65 and 67 through the network connections 62, 64 and 66. The processingelements (PE's) 63, 65 and 67 map input signal vectors to the outputdecision layer, performing such tasks as image recognition and imageparameter analysis.

A typical neural network processing element known to those skilled inthe art is shown in FIG. 4 where input vectors (X1, X2 . . . Xn) areconnected via weighting elements (W1, W2 . . . Wn) to a summing node 70.The output of node 70 is passed through a nonlinear processing element72 to produce an output signal, U. Offset or bias inputs can be added tothe inputs through weighing circuit Wo. The output signal from summingnode 70 is passed through the nonlinear element 72. The nonlinearfunction is preferably a continuous, differentiable function such as asigmoid which is typically used in neural network processing elementnodes. Neural networks used in the vehicle warning system are trained torecognize roadway hazards which the vehicle is approaching includingautomobiles, trucks, and pedestrians. Training involves providing knowninputs to the network resulting in desired output responses. The weightsare automatically adjusted based on error signal measurements until thedesired outputs are generated. Various learning algorithms may beapplied. Adaptive operation is also possible with on-line adjustment ofnetwork weights to meet imaging requirements. The neural networkembodiment of the image analysis computer 19 provides a highly parallelimage processing structure with rapid, real-time image recognitionnecessary for the Motor Vehicle Warning and Control System. Very LargeScale Integrated (VLSI) Circuit implementation of the neural processingelements permits low-cost, low-weight implementation. Also, a neuralnetwork has certain reliability advantages important in a safety warningsystem. Loss of one processing element does not necessarily result in aprocessing system failure.

In a alternate embodiment, the neural network computing network of FIG.3 can be implemented using multiple virtual processing elements 73interconnected via an image data bus 75 with an image processor 74 asshown in FIG. 5. Image data presented to the Image Processor 74 isrouted to selected virtual processing elements 73 which implement theneural network computing functions. The virtual PE's may be pipelinedprocessors to increase speed and computational efficiency.

The decision computer 23 of FIG. 1 integrates the inputs from the imageanalysis computer 19, range computer 21, digital accelerometer 45, andthe radar or lidar computer 14 to generate output warning and controlsignals. Warning signals alert the driver of impending hazards and,depending on the situation, actual vehicle control signals may begenerated to operate the vehicle in a manner that will avoid the hazardor minimize the danger to the vehicle and passengers. Control signalswill be generated to operate brake servos 33 and steering servos 36.Manual overrides are provided to ensure driver vehicle control ifnecessary.

A particularly attractive embodiment of the decision computer 23 makesuse of fuzzy logic algorithmic structures to implement the automatedcontrol and warning signal generation. Fuzzy logic is particularly wellsuited to the vehicle control problem wherein it is necessary to dealwith a multiplicity of image, motion, and environmental parameters, eachof which may extend over ranges of values and in different combinationswhich require different responses.

FIG. 6 illustrates a flow diagram for implementing a Fuzzy Logic VehicleControl and Warning signal generation system suitable for the decisioncomputer 23. The system of FIG. 6 receives inputs via the controlcomputer microprocessor 11 of FIG. 1. Inputs include image analysisoutputs, motion sensor outputs, distance measurements from radar/lidarsystems, and environmental parameters which may indicate adverse drivingconditions including rain or ice. The input signals are analyzed in apreprocessing step for hazardous conditions in the processing block 74.When a hazard is detected, the Fuzzy Associative Memory (FAM) block 76described in more detail below is activated via decision element 75. Ifno hazard is present, the system continues to analyze scanning signalsuntil a hazardous situation is encountered.

The Fuzzy Associative Memory (FAM) block 76 also receives a parameterinput file from the Detection Signal Analysis block 74. This filecontains necessary information to make control decision including, forexample, hazard location (front, back, left side, right side), hazarddistance, relative velocity, steering angle, braking pressure, weatherdata, and the presence or absence of obstructions or objects to thefront, rear, or to either side of the vehicle.

Control signals are derived using FAM's 77, 78, 79 and 80. In practice,a large number of FAM's may be used to reflect different possibledriving conditions and hazard scenarios. Each Fuzzy Associative Memorymaps input control parameter combinations to appropriate output controlsignals. The output signals are defuzzified in the control signalgenerator 81 for input to the microprocessor controller 11 of FIG. 1.This controller in turn generates control signals for steering servos,braking servos, and display and warning signals.

The FAM's operate with input signals measuring, for example, distance tothe hazard, relative velocity of the vehicle relative to the hazard andrelative acceleration between the vehicle and the hazard. Membershipfunctions for these three variables are shown in FIG. 7. The distancevariable is classified as being Very Close (VC), Close (C), Medium (M),Far (F) or Very Far (VF). Overlap between membership in the variousgrades is indicated by the overlapping trapezoids of FIG. 7. Certaindistances are in more than one membership grade, being, for example, onthe high end of being very close and the low end of being close.

Similarly, the membership functions for relative velocity grades inputsas Very Low (VL), Low (L), Medium (M), High (H) and Very High (VH) withoverlap of membership grades indicated by the intersection of membershipgrade trapezoids. Relative acceleration is graded as being eitherpositive or negative. Deceleration of the vehicle's velocity relative tothe hazard is classified as negative acceleration. Bother positive andnegative acceleration are classified as being Low (L), Medium (M) orHigh (H). Overlapping “fuzzy” membership is indicated with theoverlapping trapezoids, permitting possible membership in multiplegrades. For example, a particular velocity might have a degree ofmembership in grade “Low” of 0.2 and a degree of membership in grade“Medium” of 0.6.

Three outputs are generated from the Fuzzy Associative Memory or FAMbank: (1) Warning Level; (2) Braking Pressure and (3) Steering Angle.The fuzzy output membership functions for these signals are shown inFIG. 8. Three trapezoidal membership functions used for BrakingPressure: (1) Low Brake (LB), (2) Medium Brake (MB), and (3) High Brake(HB). Similarly, the Steering Angle is graded as Low Angle (LØ), MediumAngle (MØ), or High Angle (HØ). Steering will be right or left dependingon side obstructions, vehicles, or other conditions as indicated by thedetection signal analysis block 74 of FIG. 6. The warning level isindicated as being green, yellow, or red, depending on the danger levelpresented by the detected hazard. Continuous or discrete warnings can begenerated on the output. Possibilities include visual light indicatorsof different intensity, continuously variable audible alarms,continuously variable color indicators, or other arrangements withpossible combinations of visible and audible alarms. Warning indicatorscan be combined with actual video displays of vehicle situationsincluding hazards and nearby objects. The synthetic speech signalgenerator 27 of FIG. 1 may be used to generate synthetic speech signalsdefining spoken alarm warnings.

FIG. 9 depicts a typical FAM for generating the output control signalsfrom the input signals. Each FAM is segmented in six sections dependingon the membership grade of the acceleration variable. Interpretation ofthe FAM logic rules is straightforward. For example, if the relativeacceleration is High Positive (HP), the distance is Close (C), and therelative velocity is Medium (M), then the rule stated in the FAMrequires grading the warning as Red (R), the Brakes as Medium (MB), andthe steering as Small Angle (SØ). As a logic statement or premise, thisbecomes:

If Acceleration is High Positive (HP), Distance is Close (C), andVelocity is Medium (M), then Warning equals Red (R), Braking equalsMedium (M) and Steering Angle equals Small Angle (SØ).

As another example:

If Acceleration is Low Negative (LN), Distance is Medium (M) andVelocity is Very High (VH), then Warning equals Red, Braking equalsMedium (MB), and Steering Angle equals Small Angle (SØ).

Each premise has multiple control variables, each with possiblydifferent degrees of membership. Using fuzzy logic principles, theminimum of the truth expression for each variable can be taken as thetruth level of the premise. For example, if the membership grade foraccelerator High Positive (HP) is 0.6, for Distance Close (C) is 0.45,and for velocity medium (M) is 0.8, then the truth level for the WarningRed (R), Braking Medium (M) and Steering Angle Small (SØ) will be 0.45.

With overlapping fuzzy membership grades, more than one FAM willtypically fire in response to a given set of values for the inputcontrol variables. Each FAM that fires will yield a particular set oftruth value premises for each output variable. The result may includemultiple output memberships with different truth values. For example, itmay happen that two braking memberships result, such as Low Braking witha truth value of 0.2 and Medium Braking with a truth value of 0.6. Thecorresponding overlapping membership functions can be defuzzified usingthese values by known techniques such as the centroid method.

The FAM of FIG. 9 specifies 150 such fuzzy logic rules. Warning Levels,Braking Pressure, and Steering Angle become higher as the danger fromthe impending hazard increases. Additional FAM entries, not shown, areused to compensate for different driving conditions. For example, adifferent set of rules is used for inclement weather such as encounteredwith rain, ice or snow. Also, if side obstructions prevent steeringadjustments, different braking scenarios are necessary. Another set ofFAM logic rules is also necessary in the event of a hazard to the rearof the vehicle, simultaneous front and rear hazards, or hazardsapproaching from the right or left side. Such extensions to theteachings presented herein are described below and expand the situationsfor which the warning system offers protection in avoiding or minimizingthe effect of a collision.

The control signal generator 81 of FIG. 6 serves to defuzzify theoutputs from the Fuzzy Associative Memory. The defuzzification processconverts the output fuzzy sets into particular values that can be usedto exercise appropriate control. Various algorithms can be used todefuzzify the output including using the maximum indicated output valuein the selected membership class or the centroid method which providesoutput signals based on center of gravity calculations depending on therange of outputs indicated by the different input variables.

An important attribute of the system is the driver override featureindicated by the override input to the detection signal analysis. Thedriver override permits the driver to take control at any time bymanually braking or steering the vehicle. In practice, then, theautomated system will first warn the driver and then provide immediateautomatic corrective action if necessary. The automatic system mayoperate to control the operation of the vehicle if the driver does notproperly or quickly enough respond to indication by thewarning/indicating device controlled by the system that obstacles are inthe path of travel of the vehicle. If the warning gains the driver'sattention, the driver may then regain control with the override featureand operate the vehicle to avoid the hazard. Thus the automatic systemwill normally only apply initial corrective action with the driver thentaking control. Of course, if the driver fails to take over, theautomated system will continue to operate the vehicle to avoid orminimize the danger presented by the hazard. While manual override isprovided, the decision computer may be set to prevent the operation ofsame if it determines that a collision may occur if the driver operatesthe manual override.

FIG. 10 shows a Hazard/Object state vector used in control of the MotorVehicle Warning and Control System herein described. Each state vectorhas eight bits and represents a particular row of the possible statevectors of FIG. 12. Hazards and obstacles may occur to the front (HF),back (HB), left side (HL) or right side (HR) of the vehicle. For purposeof this discussion, a hazard is a potentially dangerous object such asanother vehicle, post, pedestrian or other obstacle when the relativemotion of the vehicle under control and the hazard could lead to acollision. An obstacle is an object to the front, rear, right side orleft side of the vehicle that might become a hazard depending on theevasive action taken by the vehicle control system to avoid a hazard. Azero (“0”) indicates no hazard or obstacle, a one (“1”) indicates thepresence of a hazard or obstacle. As indicated in the state vector,multiple hazards and/or obstacles may be present.

FIG. 11 is a Hazard Collision vector. This vector has three fieldsindicating respectively distance between the vehicle and a particularhazard, relative velocity between the vehicle and a particular hazard,and relative acceleration between the vehicle and a particular hazard.This vector is calculated for hazards detected by the image analysiscomputer 19 of FIG. 1 and various other sensors including radar/lidarsensors 14 in FIG. 1. The data in the Hazard Collision Vector is used torank hazard dangers when more than one hazard is simultaneouslydetected, and also as input to the Fuzzy Logic decision systemimplemented in decision computer 23 and described below.

FIG. 12 is a table listing various possible combinations of hazards andobstacles that may be encountered by the Motor Vehicle Warning andControl System herein described. Each row is a possible state vector oftype shown in FIG. 10. For example, state vector number 44 correspondsto a situation where there is a hazard in front of the vehicle andobstacles to the left and right of the vehicle. Thus, in this situation,it is dangerous to steer the car to the left or right to avoid thehazard. Appropriate avoidance action is this case is to slow the car tominimize the possibility of a collision with the vehicle directly infront of the controlled vehicle.

As another example from the table of FIG. 12, in state vector number 11,the hazard is to the left of the controlled vehicle. In this case, thehazard may be an approaching vehicle from the side wherein the relativemotion of the two vehicles will, if not corrected, result in acollision. The controlled vehicle is clear of obstacles to the front andback but may not turn to the right because of a potentially hazardousobstacle located there.

The state vectors of FIG. 12 are determined by the Detection SignalAnalysis block 74 of FIG. 6. The state vectors of FIG. 12 become part ofthe data file passed to the Fuzzy Associative Memory (FAM) selectionblock 76 of FIG. 6 and to the Control Signal Generator Defuzzifier 81 ofFIG. 6.

FIG. 13 is more detailed drawing of the Detection Signal Analysis Block74 of the Flow Diagram shown in FIG. 6. The more detailed flow diagramof FIG. 13 is used to set the variables in the state vector of FIG. 10and to enter parameter values in Hazard Collision vector of FIG. 11. Asshown in FIGS. 6 and 13, the Detection Signal Analysis Block 74 receivesa Sensor Input Data File from the multiple image, motion and environmentsensors of FIG. 1. This data file is used to evaluate potential hazardsand set the various control parameters needed in the Hazard/Object statevector 82 and in the Hazard Collision vector 83 of FIGS. 10 and 11respectively.

The process flow diagram of FIG. 13 first initializes the Hazard/Objectstate vector 82 and the Hazard Collision vector 83 in block 84, placingzeros in all control fields. Initial calculations are also made in thisblock using data from the sensor input data file to evaluate potentialhazards and identify objects or obstacles to the control system foralerting the driver and, if necessary, exercising direct control overthe operation of the vehicle.

Using this information, successive bits are set in the Hazard/Objectstate vector as indicated in FIG. 13. Decision element 85 will cause the“HF” bit of the Hazard/Object state vector to be set to “1” in block 86if a hazard is found in the front of the vehicle. Block 87 thencalculates the Hazard Collision vector corresponding to the frontalhazard, for entering into the Hazard Collision Vector 83 of FIG. 11.Block 11 formats those data for use in the fuzzy logic vehicle controlalgorithm hereinabove described providing numerical values for distance,relative velocity, and relative acceleration between the controlledvehicle and the frontal hazard. These numerical values are used later inthe control algorithm to rank collision hazards in the event multiple,simultaneous hazards are detected and the control system is called uponto alert the driver and possibly control the vehicle to minimizecollision impacts while dealing with multiple dangerous situations.

If no frontal hazard is detected, the flow diagram of FIG. 13 branchesaround the frontal Hazard/Object state vector operation 86 and frontalHazard Collision vector operation 87. Whether or not a frontal hazard ispresent, the flow continues to the rear hazard decision element 88 inFIG. 13. The operation here is basically identical to that describedabove for the frontal hazard calculation. If a hazard exists in back ofthe vehicle, the “HB” bit is set to logic “1” in block 89 and thecorresponding Hazard Collision vector is calculated and formatted asdescribed above for the frontal hazard situation in block 90. If nohazard exits to the rear, the blocks 89 and 90 are branched around asindicated in FIG. 13.

The same procedure is followed for hazards to the left and right ofvehicle in blocks 91 through 96 of FIG. 13. In this way, the flow fromblock 85 through 96 of FIG. 13 will set all of the hazard control bitsof the state vector 82 of FIG. 10 and provide necessary controlparameters for the Hazard Collision vector 83 of FIG. 11 for each hazarddetected by the system.

If more than one of the bits, HF, HB, HL or HR are set in the blocks 85to 96 of FIG. 13, multiple hazards exist representing a very dangeroussituation for the vehicle. The existence of multiple hazards isindicated by decision element 97 based on the values of HF, HB, HL andHR in blocks 85 to 96 of FIG. 13. If multiple hazards do exist, it isnecessary to evaluate and rank each detected hazard so that the mosteffective avoidance strategy can be adopted. The detailed collisionhazards are analyzed and ranked in block 98 of FIG. 13. Hazard rankingis achieved from the respective collision vectors of the indicatedhazards as calculated in blocks 87, 90, 93 or 96. As discussed above,the parameter values in these blocks indicate numerical values fordistance, relative velocities and relative accelerations. Using theseparameters, the time to collision can be calculated for each detectedhazard using well known kinematic equations. The most dangerous hazardthen can be determined and control signals generated accordingly.

While time to collision is an important control parameter for multiplehazards, other factors may be considered and programmed into the MotorVehicle Warning and Control System. This is especially possible withadvanced image analysis such as the neural network implementation of theimage analysis computer 19 herein before described. Using such advanced,high speed image recognition techniques will allow identifyingpedestrians, animals, particular vehicle types such as trucks or otherlarge and potentially very destructive collision objects. Specialalgorithmic sensitivity to avoid certain obstacles based on theirrespective identifications may also be programmed into processing block98 of FIG. 13.

Having ranked the collision hazards in block 98, the Hazard/Collisionstate vector 82 can be modified in block 99. This operation permitsindicating to the FAM selection block 78 of FIG. 6 which of the multipledetected hazards is currently the most dangerous. One approach is todowngrade all hazards except the most dangerous from a hazard to anobstacle in the Hazard/Collision state 82 of FIG. 10. This would ensurethat the Fuzzy Associative Memory Selection block 76 of FIG. 6 woulddirect the system to the particular FAM most responsive to the highestranking hazard as determined in processing block 98 of FIG. 13 whilestill instructing the system to avoid the other hazards.

It is also possible to set threshold levels for differences in parametervalues as calculated and compared in the ranking of collision hazards inblock 98 of FIG. 13. It may occur that multiple hazards are essentiallyof equal danger making it unwise to rank one higher than the other. Inthis case, block 99 of FIG. 13 would not upgrade one hazard overanother, but rather would use an input in the form of the Hazard/Objectstate vector 82 that ranks both as hazards, permitting selection of aFuzzy Associative Memory in block 76 of FIG. 6 that is best responsiveto the multiple hazards.

Having evaluated front, back, right side and left side hazards, the flowdiagram of FIG. 13 proceeds to set the object or obstacle bits OF, OB,OL and OR in the vector 82. Recall that front, back, left and right sideobstacles are herein defined as objects which are not currently hazardsbut may become a hazard if the wrong evasive action is taken. Examplesinclude vehicles approaching in adjacent lanes that are not on acollision course, automobiles safely behind the controlled vehicle, atree by the side of the road, and so forth. Blocks 100 through 107 setbits OF, OB, OL, and OR depending on the presence or absence of front,back, left or right objects to be avoided in controlling the vehicle.

FIG. 14 shows a more detailed flow diagram for the Fuzzy AssociativeMemory (FAM) Selection block 76 of FIG. 6. The collision vector inputscontain numerical values for relative distance, velocity, andacceleration of the vehicle and the impending hazard. Block 76 uses thisinformation as indicated in FIG. 13 to decide the respective fuzzymembership grades. Fuzzy distance membership is decided in block 109;fuzzy velocity membership is decided in block 110; and fuzzyacceleration membership is decided in block 111. Once decided, thesemembership grades serves as indices for addressing the Fuzzy AssociativeMemories (FAM's) as illustrated in FIG. 9. Membership is determined inthe respective cases by limits as indicated in FIG. 7.

The Hazard/Object state vector also serves as an index into the group ofFAMs. A simple address translation provides the actual address of theFAM locations appropriate for the detected hazard/object combinationindicated in the vector. Control signals are then directly read from theFAM ensuring rapid overall system response. Signals are immediatelygenerated to control braking, steering and warning systems as shown inFIG. 6. These output signals are likewise treated as fuzzy variableswith membership classes as shown in FIG. 7. Defuzzification takes placein processing block 81 of FIG. 6 as herein above described.

The Motor Vehicle Warning and Control System herein above described iscapable of dealing with hundreds, or even thousands, of differentcombinations of variables representing image analysis data and vehiclemotion parameters. Indeed, given the continuous nature of the variables,in the limit the number of situations is infinite. Control signalgeneration is implemented using the above described parallel imageprocessing, fuzzy logic, and fuzzy associative memories (FAM's). While acomplete logic flow diagram describing all possible flow scenarios isnot practical, it is instructive to consider the system operation for aparticular example situation. To this end, FIG. 15 illustrates thelogical system flow based on the hereinabove described embodiment forthe situation wherein the image analysis system detects a hazard infront of the controlled vehicle.

The operation of the system with this scenario is as outlined in FIG.15. The sensor input file is used to evaluate respective hazards. Theresult is the indication that a frontal hazard exists but no otherhazards are present. The hazard collision vector is prepared withnumerical values for relative distance, velocity and acceleration asindicated in FIG. 15. The system flow continues with an analysis of thepresence of objects that might become hazards depending on the evasiveaction taken by the system. There is, of course, an object in the frontof the vehicle, which is in fact the hazard of concern. An object isalso detected to the right side of the vehicle, limiting evasive actionin that direction. Using this information, the Hazard/Object vectorbecomes [10001001].

Using the collision vector for the hazard in front of the controlledvehicle, the Fuzzy Membership Grades for distance, velocity andacceleration are evaluated. Overlapping membership is possible dependingon the values for the control variables. Using the combination of theHazard/Object vector and Fuzzy Membership Grades, the FAM is accessed todetermine the “expert” driving response control signals. The FAM entriesindicate that the warning, braking, and angle steering to avoid thehazard or minimize danger to the vehicle. Defuzzification is used todetermine exact output control variable values. The steering swerve, ifany, will be to the left because of the object detected on the rightside of the vehicle. With this information, appropriate warnings anddisplays are activated and control action is taken. Even if the driverdoes not respond to the warnings, the evasive control steps will tend toreduce the danger.

In the system of FIG. 6, a different FAM is used for each state vectorof FIG. 12. Furthermore, as indicated in FIG. 9, different FAM tablesare used for different relative accelerations of the controlled vehicleand the impending hazard. There are a total of 68 state vectors in FIGS.12 and 6 different relative acceleration FAM tables in FIG. 9 yielding atotal of 408 different FAM tables. The particular FAM of FIG. 9corresponds to state vectors with a hazard in front of the vehicle onlyand no obstacles in the rear nor on at least one side. Thus this FAM maybe used with state vectors 41, 42, and 43. It can be seen that a givenFAM may be used with multiple state vectors, thereby reducing the numberof actual required Fuzzy Associative Memories or FAM's.

It is important to understand that the Motor Vehicle Warning and ControlSystem and Method herein described is based on the real time feedbackcontrol with fuzzy logic algorithms providing corrective action, theresults of which are immediately analyzed by the warning control systemusing high speed image processing based on advanced parallel computingstructures and/or neural network image analysis. The near instantaneouscontrol response required to avoid or minimize the effects of acollision are not possible without adopting these techniques. Fuzzylogic permits incremental control when necessary with continuousreal-time feedback. The results of this control are immediately sensedand further control action activated as necessary to minimize the dangerpresented by the hazard. This continuous closed loop operation closelyemulates the response of a human driver with immediate visual feedback,rapid evaluation of alternatives, and reflexive response in handling avehicle in a hazardous situation.

It is also important to note that the response rules programmed in theFAM's are “expert” driving rules for the specified conditions. Theserules are defined by expert drivers and represent the best possibledriving responses. Computer simulations and studies may also be used indefining these rules. This “Expert System” is designed to minimizedriving mistakes in hazardous situations. Note that even verbal warningscorresponding to the driving hazard/obstacle states are derived based onFAM defined expert driving responses. These warnings are delivered asdescribed above via synthetic speech system 27 of FIG. 1. Thus thedriver has the assistance of an on-board, real-time expert speaking tohim or her and advising on the optimum driving response to a givenroadway condition.

A further extension of the described system is responsive to visually orelectronic detectable road markers such as lane markers, safe speedmarkers, curve warnings, or other hazard indicating devices installedalong or in the roadway. The same system herein above described can beresponsive to signals detected from such warnings and integrate thisinformation into the overall vehicle control system.

In a modified form of the invention, it is noted that system 10 may alsoperform as a navigational computer informing the driver of the motorvehicle containing same of the location of the vehicle by controllingthe display 32 to cause it to display characters describing suchlocation and/or a map showing the road or street along which the vehicleis travelling and its location and direction of travel there along bymeans of an indicia such as an arrow. The map may graphically or bymeans of characters include auxiliary information such as towns andcities along the route of travel, distances thereto, alternate routes oftravel, road conditions, information on traffic density, hazardousconditions, weather ahead, sightseeing information and other informationderived via short wave or other receiving or input means which outputsdigital codes to RAM memory 12 and/or other computer or microprocessor11. Such information may be derived via earth satellite short wavetransmission and/or local or roadside radio transmitters as the vehicleapproaches and passes same and/or may be input via wire or short wave toa short wave receiver of the vehicle, such as its audio radio, receiveror an auxiliary receiver connected (via an analog-to-digital converter)to computer 11 via an input bus (not shown).

The memories 12 and 13 or other memories may also be programmed withtrip or travel data derived via short wave, telephone line, microwavesatellite or other communication system connected to a remote computeror by a select pluggable memory or recorder output. Vehicle instantlocation data codes may be received via satellite location or electronictriangulation and the codes generated may be employed to properly accessmap defining graphics data and to effect the display of the proper mapgraphics on the heads-up or video display 32.

A keyboard 82 and/or microphone (located, for example, in the steeringwheel or steering wheel hub) of the vehicle and a speech recognitioncomputer such as computer 25 may be employed by the driver to generatecommand control signals for controlling the trip or navigationalcomputer and effecting the display and/or playback of synthetic speechof select information on the location, direction of travel, distances toselect locations, towns or cities, map information or other informationas defined above.

In yet another form of the invention, the memory 20 of the imageanalyzing computer 19 and/or an auxiliary memory therefor may containimage data derived from the output of a television camera on a vehicletravelling the same road, roads or route travelled by the driven vehiclecontaining system 10. Such image data may be derived from archivalmemory once the expected route or routes of travel is known, whichachieved memory data was generated by computer processing the output ofTV camera 16 of system 10 during previous travel of the vehicle alongthe same route and/or from TV scannings of other vehicles. Suchpreviously generated image signal data may be utilized to improve oreffect proper operation of system 10 by providing data on stationaryobjects and background, or road images along the route of travel.

Thus computer 11 may have (a) a microphone and analog to digitalconverter of speech signals connected thereto as well as (b) a shortwave receiver of data and (c) an input keyboard as described.

Another form of the invention involves short wave (for example,microwave or infra-red) communication between two or more vehiclescontaining respective systems 10 to effect cooperative control functionsto be performed by the computers of both vehicles. A short wave radiotransmitter 86 is shown in FIG. 1 connected to microprocessor 11 toreceive digital codes from the decision computer 23 which codes aregenerated when a hazardous driving or road condition is detected asdescribed and may involve a collision with a vehicle travelling in thesame or opposite direction as the vehicle containing system 10 whichdetects such condition. Such code signals sent by short wave microwave,radar or infra-red transmitter-receivers of either or both vehiclesand/or other vehicles in the vicinity of the developing hazard may beemployed on receipt to warn the driver of the other vehicle(s) of thehazardous condition with suddenly generated synthetic speech, flashinglights, tones, etc. and/or effect an automatic vehicle control operationsuch as an automatic braking and/or steering operation, as described, toavoid or reduce the effects of a collision. The infra-red communicationsystem may involve code pulsed infra-red diodes or lasers and solidstate receivers of infra-red light mounted on the front and rear bumpersof the vehicles.

It is also noted that system 10 may be employed with suitable softwareas described above, or with additional sensors or sensing systems addedto the system to sense traffic lane times along roads and highways,active and/or passive signal or code generators and short-wavetransmitters buried in the highway and/or at the side of the roadtravelled and/or supported by other vehicles, to automatically operatethe vehicle containing such computerized system during the normal travelof such vehicle between two locations and/or destinations. For example,select highways or select sections of a highway may be designed andoperable to accommodate (only) vehicles which are equipped with system10 which is operable to steer and control the speed of the vehicle inaccordance with control signals generated by the decision computer 23when it is specially programmed to guide and control the speed of thevehicle in its travel along the select highway or road. To supplementthe signals generated by the image analyzing computer 19, or as areplacement therefor, an auxiliary computer, not shown, may be providedconnected to the control computer 11 and operable to receive and analyzeinformation signals or codes generated as a result of digitizing theoutput(s) of one or more sensors on the vehicle sensing (a) highwaymarker or lane delineating lines, (b) curb and/or divider markings, (c)embedded or roadside code generators, or (d) electro-optically scannableindicia or reflectors along and/or at the side of the road or acombination thereof. The short wave receiver 84 may receiveradio-frequency codes generated locally as the vehicle passes while oneor more electro-optical scanning systems employing solid state lasersand photodetectors of the reflected laser light may be employed toprovide such coded information which is processed by computer 19 or theauxiliary computer to provide vehicle control or operational signalswhich may be used per se or by the decision computer 23 to control andmaintain control of the vehicle to keep it travelling in a select laneand at a select speed in accordance with the set speed for the highwayor the select lane thereof along which the vehicle is travelling and/orthe speed of other vehicles ahead of the computer controlled vehiclecontaining system 10.

A further enhancement of the herein defined automated vehicle warningsystem makes use of a separate driver monitoring computer to constantlymonitor driver actions and reactions while operating the vehicle. Thistype of monitoring is especially helpful in determining driver fatigueor detecting erratic driving patterns caused for example, from drivingwhile intoxicated or under the influence of drugs. Erratic drivingpatterns may include swerving in steering of the vehicle, uneven orunnatural acceleration or deceleration, combinations of unusual orunnatural driving patterns, driving much slower or faster than othervehicles around the automobile being monitored, unnatural sequences ofexercising control over the vehicle such as alternate braking andacceleration, braking or stopping in a flowing traffic stream, orexcessive acceleration. Also, driving patterns inconsistent withsurrounding vehicle motion can be detected such as any action by thedriver that increases rather than decreases the possibility of acollision in a dangerous or hazardous situation. A separate drivermonitoring system can detect all of these situations and respond bywarning the driver or, if necessary, activating the automated vehiclecontrol system.

The motor vehicle warning and control system can warn other vehicles ofan impending or detected possible collision by flashing exterior warninglights and/or sounding audible alarms including the horn. The system mayalso warn other vehicles via radio transmission which activates warningsin adjacent vehicles of dangerous situations. Drivers of other vehiclescan then be warned by audible or visual warning devices and/or displaysand can take necessary evasive action. The radio signal can also alertpolice or highway patrolmen of dangerous driving patterns by identifyingthe vehicle. As a further extension, the vehicle may have an electroniclocation system such as satellite Global Position System (GPS)electronics permitting precision vehicle location, which information canbe transmitted with the hazard warning signals, permitting lawenforcement and roadway safety personnel to precisely locate the vehicledetected as being in a hazardous situation caused by the driver or otherconditions.

A further enhancement of the vehicle warning and control system andmethod disclosed herein makes use of a recorder to record the lastseveral minutes of driving action for future analysis. Such recordingspermit reconstruction of events leading up to collision permitting moreaccurate determination of causes including fault.

1. A system for operating and controlling a motor vehicle having apower-drive system and controls including an accelerator, a brake, and asteering system comprising: (a) a ranging device supported by thevehicle, directed toward the front of the vehicle, and structured togenerate first signals indicating the distance to and relative motion ofsome objects in front of the vehicle; (b) a detector supported by thevehicle, directed away from the vehicle in a direction other than thefront of the vehicle, and structured to generate second signalsidentifying whether there is an object in said other direction; and (c)a fuzzy logic-based computer having a memory containing a plurality ofsets of fuzzy inference vehicle control rules, each rule defining acoordinated combination of changes to the vehicle's steering andacceleration; (d) wherein the computer is coupled to the ranging deviceand the detector and structured to use the first and second signals toselect and reproduce from the memory a selected one of the plurality ofsets of rules and to apply the selected set of rules to derive commandsignals with fuzzy logic; (e) wherein the computer is electricallycoupled to the controls; (f) whereby the command signals are applied tocontrol the accelerator, brake, and steering system of the vehicle in acoordinated way to attempt to avoid collisions between the vehicle andobjects in its path of travel without taking evasive action that wouldcause a collision with objects detected in said other direction.
 2. Asystem in accordance with claim 1 further comprising a visual displayinside the vehicle coupled to the ranging device and driven by the firstsignals to generate symbols representative of objects in the path of thevehicle.
 3. A system in accordance with claim 2 wherein the visualdisplay comprises a heads-up display aimed to project images ofintelligible information on a front windshield of the vehicle.
 4. Asystem in accordance with claim 1 further comprising a synthetic speechgenerating system coupled to the computer and driven by the commandsignals to generate sounds of select words of speech.
 5. A system inaccordance with claim 1 further comprising a warning device coupled tothe ranging device and driven by the first signals to generate a warningsignal perceptible to a human when one of the objects is in the path ofthe vehicle, and wherein the computer is timed to control the operationof the vehicle only if the controls of the vehicle are not first alteredsufficiently to avoid a collision with the object in response toindication by the warning device that an obstacle is in the path oftravel of the vehicle.
 6. A system in accordance with claim 1 whereinthe ranging device comprises an image-generating camera.
 7. A system inaccordance with claim 1 wherein the ranging device comprises aradar-based ranging system.
 8. A system in accordance with claim 1further comprising an override controller coupled to the computer so as,when activated by a driver, to prevent the command signals fromcontrolling the motor vehicle.